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1  Summary 

System  identification  (SID)  is  the  task  of  deducing  the  internal  dynamics  of  a  black-box  system  solely 
from  observations  of  its  outputs.  This  is  an  essential  first  step  in  a  variety  of  engineering  problems; 
most  traditional  control-theoretic  methods,  for  example,  require  an  accurate  ordinary  differential 
equation  (ODE)  model.  Accuracy  is  not  the  only  requirement,  however;  for  efficiency  reasons, 
engineers  work  hard  to  construct  minimal  models  —  ODEs  that  ignore  unimportant  detaU  and  capture 
only  the  behavior  that  is  important  for  the  task  at  hand.  This  tandem  of  accuracy  and  abstraction  is  a 
subtle  and  difficult  part  of  an  engineer's  training  and  practice. 

The  computer  program  pret,  the  topic  of  this  report,  automates  the  process  described  in  the  previous 
paragraph  by  building  an  artificial  intelligence  (AI)  layer  around  a  set  of  traditional  system 
identification  techniques.  This  AI  layer  executes  many  of  the  high-level  parts  of  the  SID  process  that 
are  normally  performed  by  a  human  expert,  intelligently  assessing  the  task  at  hand  and  then  reasoning ' 
from  that  information  to  automatically  choose,  invoke,  and  interpret  the  results  of  appropriate  lower- 
level  techniques.  These  tactics  guide  pret  quickly  and  accurately  to  the  minimal  ODE  that  accounts 
for  the  important  behavior  of  the  system. 

The  ultimate  goal  of  this  project  is  to  build  a  tool  that  can  automatically  construct  models  of  high¬ 
dimensional,  nonlinear,  black-box  systems,  drawn  from  any  domain  that  admits  ODE  models.  TTie 
objective  of  this  project  is  to  effectively  automate  existing  SID  techniques,  not  invent  new  ones;  even 
so,  this  is  an  extremely  ambitious  goal,  as  nonlinear  system  identification  is  a  state-ofrthe-art  research 
area.  As  an  automated  tool  that  accomplishes  this,  pret  will  be  of  obvious  practical  importance  in 
science  and  engineering:  as  a  corroborator  of  existing  models  and  designs,  as  a  medium  within  which 
to  instruct  newcomers,  and  as  an  intelligent  assistant,  whose  aid  will  allow  more  time  and  creative 
thought  to  be  devoted  to  other  demanding  tasks. 

2  FY98  Progress 

One  of  the  core  technical  challenges  in  this  project  is  to  choose  and  orchestrate  appropriate  reasoning 
levels  and  techniques  for  each  stage  of  the  modeling  task.  This  year,  we  have  completed  the 
construction  of  a  specialized  first-order  logic  system  that  performs  this  task  effectively  and  efficiently. 
This  custom  logic  system  uses  SLD  resolution  and  a  mathematical  domain  theory  that  ranges  from 
trigonometry  through  calculus  to  nonlinear  dynamics  in  order  to  automatically  generate  and  test  ODEs 
against  a  set  of  observations.  The  reasoning  processes  that  pret  uses  in  the  course  of  the  modeling 
process,  like  those  used  by  human  experts,  are  mostly  qualitative  and  highly  heterogeneous.  The 
logic  system  is  designed  to  take  maximum  advantage  of  this;  it  uses  declarative  meta  control  to  adapt 
dynamically  to  the  current  state  of  the  theorem  prover,  thereby  assuring  that  the  cheapest  and  most 
appropriate  (i.e.,  most  likely  to  yield  useful  information)  form  of  reasoning  is  used  at  all  times.  A 


“sparse”  truth-maintenance  system  (STMS)  adds  to  this  efficiency,  letting  the  logic  system  save  and 
reuse  information  that  is  useful  in  the  context  of  pret's  particular  reasoning  task  —  information  that 
would  be  lost  in  a  garden-variety  theorem  prover.  Finally,  static  abstraction  levels  enforce  the 
minimality  constraint,  assuring  that  pret  does  no  more  work  than  necessary  to  model  the  target 
system. 

Another  important  and  difficult  part  of  this  project  is  the  choice  of  the  internal  representation.  We 
chose  ODEs  as  pret's  target  models  because  they  are  both  general  and  powerful,  but  we  have 
recently  begun  to  suspect  that  they  are  not  necessarily  the  best  internal  representation.  After  thinking 
hard  about  this  issue  over  the  last  year,  we  have  decided  to  retrofit  pret  to  use  bond  graphs  —  a 
generalized  representation  of  a  network  of  power  relationships  that  is  a  common  and  powerful  tool  in 
an  expert  control  theorist's  arsenal  —  as  the  primary  internal  representation.  The  important  notion 
that  the  same  ODE  may  model  different  systems  in  different  domains  is  captured  naturally  in  bond 
graphs;  their  inherent  abstraction  brings  out  similarities  and  analogies  —  e.g.,  that  springs  and 
capacitors  can  play  analogous  roles  in  ODE  models  of  mechanical  and  electronic  systems.  This 
feature  should  make  it  easy  to  add  intelligence  to  pret's  model  generator,  which  currently  enumerates 
models  in  an  unintelligent  (and  exponentially)  complex  manner.  Bond-graph  properties  also  induce  a 
neat  and  obvious  structure  for  the  syntax  and  semantics  for  sensor/actuator  specification  —  another 
hard  and  important  problem  on  our  remaining  to-do  list. 

We  have  also  made  progress  on  variety  of  other  fronts.  Among  other  things,  the  graphical  user 
interface  (GUI)  is  now  functional,  and  we  added  a  variety  of  domain  mles  to  the  rulebase,  as  inspired 
(and  demanded)  by  new  examples.  Implementing  automated  VO  interactions  with  the  real  world, 
however,  continues  to  give  us  trouble;  the  problems  involved  are  much  more  difficult  than  we  had 
anticipated.  This  is  not  completely  su^rising,  as  we  are  proposing  to  automate  one  of  the  harder 
branches  of  nonlinear  control  theory  in  this  section  of  the  project:  automatically  determining  what 
kinds  of  tests  one  can  perform  on  a  system,  what  kinds  of  .  results  one  can  obtain,  and  what 
conclusions  one  may  draw  from  them.  ' 

3  Accomplishments 

Of  the  eleven*  action  items  for  the  time  remaining  on  this  contract  that  were  listed  on  pp  4 — 5  of  our 
FY97  report,  we  have  finished  five  and  made  significant  progress  on  five,  as  summarized  below. 

•  The  logic  system  that  forms  pret's  core,  which  uses  SLD  resolution,  static  abstraction  levels, 
dynamic  meta  control,  constraint  propagation,  and  a  “sparse”  truth-maintenance  system 
(STMS)  —  a  significant  innovation  —  to  build  and  check  ODE  models,  is  now  complete  and 
case-hardened. 

In  FY98,  Reinhard  Stolle,  the  author  of  this  logic  system,  defended  his  Ph.D.,  received 
offers  for  two  tenure-track  jobs  and  a  variety  of  postdoc  positions,  and  published  two  papers, 
one  in  the  highly  prestigious  —  and  competitive  —  National  Conference  on  AI  (AAAI).  His 
paper  on  the  dynamic  meta  control  system  is  in  review  at  Knowledge  and  Information 
Systems,  and  a  companion  paper  on  the  STMS  is  in  preparation.  He  will  be  taking  a  postdoc 
at  Stanford  in  January  1999. 

•  pret's  graphical  user  interface  has  been  completed  and  tested. 

Apollo  Hogan,  who  designed  and  implemented  that  GUI  and  worked  closely  with  Stolle  on 
the  meta  control  and  STMS  systems,  is  a  co-author  on  the  Knowledge  and  Information 


'  There  were  actually  twelve  bullets  on  that  list,  but  one  of  those  tasks  —  augmentation  of  the  rule  base  —  is  by 
definition  ongoing. 


Systems  paper  listed  above.  In  August  1998,  he  will  be  entering  the  Ph.D.  program  at 
Berkeley  in  set-theoretic  topology. 

•  We  spent  a  great  deal  of  time  thinking  about  and  redesigning  pret's  internal  representation  to 
use  bond  graphs  instead  of  ODEs;  this  adaptation,  which  has  not  yet  been  implemented,  will 
have  sweeping  positive  effects  on  the  remaining  action  items,  as  detailed  in  section  5  of  this 
report. 


4  Transitions 

PRET  has  now  been  developed  to  the  point  where  outsiders  can  use  it,  but  only  with  significant 
assistance  from  one  of  us.  Interactions  with  application  analysts  have  given  us  useful  insights  and 
suggestions  (e.g.,  the  GUI)  and  have  forced  us  to  augment,  polish  and  bomb-proof  the  code.  Three 
key  example  applications,  in  particular,  have  driven  and  will  continue  to  drive  pret's  development: 

•  A  radio-controlled  car  used  in  a  soccer-playing  robot  system  by  Dinesh  Pai's  group  at  the 
University  of  British  Columbia 

•  An  eyeglass  lens  cutting  machine  in  David  Trumpet's  lab  at  MIT 

•  A  model  of  a  ship-mounted  crane  constructed  by  Randall  Tagg  at  the  University  of  Colorado 
at  Denver 

pret's  results  on  the  R/C  car  are  complete;  see  our  FY97  report  and  the  accompanying  quadchart 
materials.  The  lens-cutting  machine  and  the  on-ship  crane  are  new  endeavors  this  year.  The  crane,  in 
particular,  has  obvious  and  significant  naval  applications  —  positioning  and  stabilizing  a  load 
swinging  from  a  deck-mounted  crane  in  rough  seas  is  a  difficult  control  problem,  and  an  accurate 
ODE  model  of  the  crane-ship-load  system  is  essential  to  its  solutibn. 

Our  interactions  with  human  experts  have  exposed  a  variety  of  subtle  and  difficult  AI  issues.  For 
example,  one  analyst  intuitively  and  immediately  rejected  a  model  that  matched  his  spec  perfectly  — 
because  it  did  not  match  some  of  his  implicit  knowledge  about  the  system.  Frames  of  reference  in 
which  to  reason  have  also  caused  some  difficulties:  pret  is  not  equipped  to  choose  the  “best” 
reference  frame  for  a  given  problem  —  a  choice  that  is  a  large  part  of  an  engineering  or  science 
education,  and  that  is  simply  too  hard  to  automate. 

pret  is  still  at  the  stage  where  interactions  like  this  are  far  more  useful  to  us  than  tO‘  our  users.  We 
hope  to  begin  changing  that  this  year,  and  we  have  some  preliminary  evidence  that  we  ^ill  indeed  be 
able  to  do  so.  In  the  past,  the  goal  of  these  exercises  has  been  to  duplicate  (and  thereby  verify)  a 
model  found  by  a  human  analyst;  in  the  case  described  in  the  previous  paragraph,  however,  pret's 
model  was  actually  better,  which  suggests  that  we  can  indeed  make  useful  contributions  to  the 
modeling  art. 

We  have  also  collaborated  with  the  Information  Technology  group  at  NIST/NO AA  in  Boulder  in 
order  to  develop  an  Al-adapted  global  optimization  tool  that  combines  qualitative  reasoning  and  local 
numerical  methods;  see  the  Annals  of  Mathematics  and  Artificial  Intelligence  paper  listed  below. 

5  Plans  for  next  year 

This  work  plan  is  designed  to  cover  the  next  12  months,  but  the  contract  ends  on  31  March  1999,  and 
we  will  not  finish  all  of  these  items  by  then.  Primary  personnel  for  this  period  include  one  Ph.D. 
student  (Matt  Easley)  and  the  PI;  Stolle  has  agreed  to  stay  on  as  a  postdoc  for  the  fall  semester  in 
order  to  help,  us  finish  some  of  the  tasks  listed  under  item  1  below. 


1.  Bond  graphs: 

•  Retool  PRET  to  use  bond  graphs  as  an  internal  representation 

•  Begin  working  out  how  to  exploit  this  representation  to  improve  the  model  generator 

•  Test  the  power  of  these  improvements  by  implementing  a  new  domain:  mechatronics 

•  Begin  working  out  how  best  to  structure  the  syntax,  semantics,  and  representation  of  the 
information  that  pret  receives  about  the  sensors  and  sends  to  the  actuators 

•  Begin  automating  pret 's  output  interactions 

•  Easley  will  defend  his  Ph.D.  proposal 

2.  Examples 

•  Build  and  test  a  simple  mechanical  system  with  input/output  capabilities  —  probably  a 
spring-mass  oscillator  on  an  air  track 

•  Continue  working  with  David  Trumper's  group  at  MIT,  who  are  performing  system 
identification  on  a  commercial  eyeglass  lens-cutting  machine,  and  attempt  to  reproduce 
their  results  automatically 

•  Continue  working  with  Randall  Tagg  at  the  University  of  Colorado  at  Denver,  applying 
PRET  to  the  on-ship  crane  mockup  in  his  lab 

3.  Rulebase:  (ongoing) 

•  Continue  adding  rules  as  pret  is  tested  on  more  examples  in  different  domains 


6  Publications  on  This  Project 

\ 

All  are  full-length,  refereed  papers. 

•  E.  Bradley,  A.  O'Gallagher,  and  J.  Rogers,  “Global  Solutions  for  Nonlinear  Systems  using 
Qualitative  Reasoning,”  Anna/5  of  Mathematics  and  Artificial  Intelligence  (1998) 

•  E.  Bradley  and  M.  Easley,  “Reasoning  About  Sensor  Data  for  Automated  System 

Identification,”  Intelligent  Data  Analysis:  An  International  Journal,  vol.  2  no.  2,  Elsevier 
Science  (1998)  HTrp://www.ELSEViER.C0M/LC)CATE/iDA  [Published  electronically  and  archived 
on  CD;  neither  format  includes  page  numbers]  ^ 


•  R.  Stolle  and  E.  Bradley,  “Multimodal  Reasoning  about  Physical  Systems”  National 
Conference  on  Artificial  Intelligence  (AAAI),  Madison  WI;  July  1998  [Expanded  version  of 
the  paper  listed  below ] 


R.  Stolle  and  E.  Bradley,  “Multimodal  Reasoning  about  Physical  Systems,”  AAAI  Spring 
Symposium  on  Multimodal  Reasoning',  Stanford  CA;  March  1998 


